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ABSTRACT 



The Secondary Analysis of TIMSS (Third International 
Mathematics and Science Study) Video Data study used TIMSS Videotape 
Classroom Study data and vPrism software to achieve methodological and 
substantive findings on teacher quality, instructional practices, and 
classroom interactions. It discussed methodological and technical issues 
arising during secondary analysis of existing data sets using specific 
software applications. The paper examines three methodological findings from 
using the video data and software to explore teachable moments: the technical 
dictates of how an utterance is defined and operationalized for coding may 
impact a study's unit of analysis and overall design; coders coding in pairs 
or groups may be more reliable; and it is important to do the empirical work 
of developing a study design, codes, and coding and analysis protocols and to 
critically reflect on assumptions and processes involved. This secondary 
analysis resulted in increased awareness of the significant interplay between 
the unit of analysis, the coding unit, and the software application. The 
study showed that it was fruitful to use the TIMSS video data set for 
secondary analysis. There appeared to be an instructional practice called a 
"teachable moment" as defined, and it was possible to develop, apply, and 
test codes and their reliability. (SM) 
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Introduction and Background 



Since 1994, substantial interest has been directed toward large-scale, quantitative studies 
that use video as a data source. One such study, the Third International Mathematics and 
Science Study (TIMSS) Videotape Classroom Study (Stigler, Gonzales, Kawanaka, 
Knoll, & Serrano, 1999), videotaped over 225 8th grade mathematics lessons in 
Germany, Japan, and the United States. The Third International Mathematics and 
Science Study - Repeat Videotape Classroom Study (TIMSS-R) (TEMSS-R, 
forthcoming) has gathered video data of over 1100 mathematics and science lessons in 
Australia, the Czech Republic, Hong Kong, Japan, the Netherlands, Switzerland, and the 
United States. Based on nationally-representative samples of 8th grade classrooms, these 
video data sets offer unique information about teaching and learning within and across 
nations. They also draw upon new video format and analysis technologies which make it 
easier to gather, store, and manipulate large quantities of video data. The result has been 
valid and reliable statistical analyses of classroom interactions and practices, within 
cross-cultural, comparative contexts. Also, the TIMSS Videotape Classroom studies 
have provided, and will continue to provide, opportunities for secondary analyses. 

Using video as a data source, along with specific software applications for manipulating 
it, will likely become more common in coming years for large-scale survey-type research 
and smaller scale ethnographic or case studies alike. For this reason, it seems important 
to learn more about the substantive benefits of these new types of research approaches 
and their methodological and technical intricacies. 

The study we report on here - the Secondary Analysis of TIMSS Video Data Study 
(SATV) — was undertaken by the Educational Statistics Services Institute (ESSI) of the 
American Institutes for Research (AIR) for the National Center for Education Statistics 
(NCES). The SATV study’s broad charge was to use the TIMSS Videotape Classroom 
Study data set and the vPrism software application (developed for use by the original 
TIMSS Videotape Classroom Study) to achieve both methodological and substantive 



1 Because of the study’s size and preliminary nature, it would be inappropriate to generalize results to other video 
studies using similar methods - especially ones at a larger scale. While results do point to areas of interest, any 
questions that the SATV study raises warrant additional exploration and research in order to further develop the current 
knowledge base regarding the use of digital video in research. 

2 Authors can be contacted at sarafeh@air.org, bsmerdon@air.org, and ssnow@air.org. 
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findings. The study sought to shed light on substantive issues related to teacher quality, 
instructional practices, and classroom interactions. In the process, however, it also 
explored methodological and technical issues that might arise when doing secondary 
analysis of an existing data set using a particular software application. In this paper, we 
report on three methodological findings that resulted from using the TIMSS video data 
set and vPrism 3 software application to explore “teachable moments.” 

The first finding concerns the functional interplay between our study’s unit of analysis 
(the utterance) and unit of coding (transcript unit) based on the technical requirements of 
the vPrism software. The second finding relates to coding procedures and how coding 
reliability might be increased by asking coders to code as pairs or teams rather than 
individually. We discuss these findings in more detail below. However, the broad 
finding of the SATV study is the importance of doing the empirical work of developing a 
study design, codes, and coding and analysis protocols and, more importantly, of 
critically reflecting on the assumptions and processes involved. The attention to detail 
and opportunity for self-reflexive analysis that secondary analyses of video data and data 
sets can offer are important not only for the new substantive analyses they can provide, 
but also for the conceptual, methodological, and technical insights that can result from 
them. 

Methods 

Teachable Moments 

Figure 1. Representation of Teachable Moment and Codes 




The SATV Study set out to identify and broadly characterize components of “teachable 
moments” using quantitative methods and the TIMSS video data. The code development 
team defined teachable moments as the set of behaviors within a lesson that indicated 
students are ripe for, or receptive to, learning because they express confusion, 



3 vPrism is a trademarked, multimedia database that was developed to manage and analyze the 250-plus hours of video 
gathered for the initial TIMSS Videotape Classroom study. vPrism is one of many software applications that links 
digital video with transcript and other text or object information. Currently sold and supported by Lesson Lab, Inc., 
contact information for the software and can be found through their website www.lessonlab.com . 
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